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This page describes the data which is stored in 
BLACK HOLE , where it comes from and how 
much of it there is. Some of the information is 
too highly classified to be contained here, or 
falls under the Applied Research intellectual 
property restrictions. Nonetheless, as much as 
possible is published here. 
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What data BLACK 
HOLE contains 



r editl Types of data 

The events created cover webmail, email 
transfers, ftp, chat, internet browsing, website 
logins, vbulletin web fora, web cams, gaming, 
social networking — and the list is growing. 

The data is primarily generated by B13B high-rate bearer processing modules, and so the exact types 
depend on which modules are running at the time. 

f editl Classification of the data 




BLACK HOLE: The Knowledge 



• What it is 

• Where it came from 

• What is in it 

o What data it contains 
o How much data there is 

• How to use it 

o How to connect to it 
o How to find the file you want 

• Who uses it. and why 

o Applied Reseach uses 
o SIGINT Development uses 
o IPT uses 

• How it is built 

o Logical connectivity 
o Physical system 
o Security issues 

• What could happen to it 

o Applied Research enhancements 
o SIGINT Development intentions 
o Engineering interest 

• What we like about it 

o Good things 

o Things that could be better 
o Exploding myths 
o Wish list 

v d e 



The classification of the data in BLACK HOLE ranges from SECRET STRAP 1 to TOP SECRET 
STRAP2 UK EYES ONLY. Most data should be covered by TOP SECRET STRAP2 and there may, 



inadvertently, be ECI data (but this is not expected, as we are not actively doing things covered by ECI). 
The data includes fields which have been produced by enrichment (such as, for example, target details) 
and this can increase the classification. 



leditl Where the data in BLACK HOLE comes from 

Take a look under the covers of BLACK HOLE to find out where the data comes from. 

leditl How much data there is 

r editl Volumes in BLACK HOLE 

The total amount of (compressed) data entering BLACK HOLE is approximately 200GB/day. 

r editl Volumes in the datamarts 



Prototype 


Current 

retention 

time/ 

months 


Current 
database 
size / TB 


Estimated size 
for six months 
retention / TB 


Current 
database size 
/ billion rows 


Estimated 
database size for 6 
months retention / 
billion rows 


MUTANT 

BROTH 


4 


7.7 


11.55 


21.7 


32.55 


KARMA 

POLICE 


3 


6.8 


13.6 


17.8 


35.6 


MEMORY 

HOLE 


0.5 


0.6 


7.2 


2.2 


26.4 


AUTOASSOC 


6 


0.1 


0.1 


11 


11 


SOCIAL 

ANIMAL 


1 


1.6 


9.6 


1 


6 


HRMAP 


3 


7 


14 


16 


32 


INFINITE 

MONKEYS 


6 


0.02 


0.02 


0.2 


0.2 


Total 




23.82 


56.07 


69.9 


143.75 



Please note - all of these values are approximate. 
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